This file contains robustness and sensitivity checks for models evaluating the imapact of agriculutral diversification on U.S. crop production. To reduce the number of models evaluated only national level models for corn using the richness diversity index are examined in this document.

Examining Errors

While model fit statistics and scatter plots of observed and predicted values generally indicate the model fits the data well there are a high number of outliers that can be seen in the predictive p-value plot (0s and 1s). Here we plot the errors (observed yield minus mean predicted yield) in order to identify any trends associated with these outliers. Positive values indicate underestimation of yields and negative values indicate overestimation of yields. As shown below the majority of errors fall near 0. High errors do not appear to occur consistently in the same region over time, but do seem to show highly localized clustering that may be indicative of localized extreme natural disaster events not captured in the model. For example overestimation of yields along small sections of the Mississippi, Ohio, and Missouri rivers and along the eastern coastline can be seen in several years corresponding to years with hurricanes (2011) and low river levels induced by severe drought (2012). These suggest that our model is capturing global and regional variability in annual average trends (particularly climate and weather trends), but does not completely account for short-term (sub-annual) localized extreme weather conditions and other hazards (pest outbreak, wildfire, etc…).

CROSS SECTIONAL MODELS

Individual cross-sectional models are run to illustrate how much the results from the panel model are influenced by differences across space and how variable these results are over time. Results for 2010, 2013, and 2016 are shown below. The cross sectional models suggest that much of the diversity effect does arise from differences that occur across space, but that this is also variable across years. For example, there is no significant effect of diversity in 2013, a year of widespread severe drought, while there is a clear effect of diversity in 2016. Generally, the climate variable functional responses appear to be consistent across years with the effect of precipitation being most variable.

[[1]]


Table: Summary Table of Model Estimates

                                                 mean            sd   0.025quant     0.5quant    0.975quant
----------------------------------------  -----------  ------------  -----------  -----------  ------------
(Intercept)                                    4.6705        0.0241       4.6228       4.6707        4.7174
PERC_IRR                                       0.0081        0.0008       0.0066       0.0081        0.0095
ACRES                                          0.0000        0.0000       0.0000       0.0000        0.0000
Precision for the Gaussian observations    19275.7019    18651.2583    1310.9579   13807.7564    68572.9543
Precision for TP                             295.2861      211.9585      68.6953     239.5979      853.1462
Precision for SDD                            189.5073      108.3873      61.0438     163.4012      471.0844
Precision for GDD                              9.0408        3.1157       4.5224       8.5074       16.6204
Precision for RICH                         25229.7911   163647.5896     482.5656    4887.7963   165314.9398
Precision for CNTY                            10.9191        0.5765       9.8295      10.9040       12.0968
Phi for CNTY                                   0.8812        0.0154       0.8483       0.8822        0.9086
Precision for AERCODE.id                   18147.6320    18181.7137    1239.3959   12762.1212    66186.6441
Precision for AERCODE.id2                  18149.0240    18207.4138    1254.8587   12761.1995    66202.9458

[[2]]


Table: Model Diagnostic Metrics

       DIC        CPO     MSE         R2
----------  ---------  ------  ---------
 -9916.024   3613.568   3e-07   0.998486
[[1]]


[[2]]

[[1]]

[[1]]

[[1]]

[[1]]

[[1]]


Table: Summary Table of Model Estimates

                                                 mean           sd   0.025quant     0.5quant    0.975quant
----------------------------------------  -----------  -----------  -----------  -----------  ------------
(Intercept)                                    4.7209       0.0243       4.6733       4.7208        4.7686
PERC_IRR                                       0.0060       0.0006       0.0047       0.0060        0.0073
ACRES                                          0.0000       0.0000       0.0000       0.0000        0.0000
Precision for the Gaussian observations    20027.9616   18942.0419    1593.5696   14586.5197    69618.4852
Precision for TP                              44.3463      19.8627      16.7663      40.7475       93.0678
Precision for SDD                             79.7257      31.9101      34.6633      74.0352      158.0983
Precision for GDD                             60.4597      22.6001      27.9507      56.5957      115.4720
Precision for RICH                         19704.9624   69437.9931     786.9984    6129.4367   121424.7542
Precision for CNTY                            13.3839       0.8447      11.7857      13.3626       15.1106
Phi for CNTY                                   0.9639       0.0130       0.9328       0.9659        0.9834
Precision for AERCODE.id                   18728.3099   18564.2782    1230.6105   13226.9280    67745.6135
Precision for AERCODE.id2                   2371.9523    3171.2091     239.0234    1426.4721    10299.2692

[[2]]


Table: Model Diagnostic Metrics

       DIC        CPO     MSE         R2
----------  ---------  ------  ---------
 -9014.273   3122.628   4e-07   0.997685
[[1]]


[[2]]

[[1]]

[[1]]

[[1]]

[[1]]

[[1]]


Table: Summary Table of Model Estimates

                                                 mean           sd   0.025quant     0.5quant    0.975quant
----------------------------------------  -----------  -----------  -----------  -----------  ------------
(Intercept)                                    4.8295       0.0226       4.7852       4.8295        4.8738
PERC_IRR                                       0.0049       0.0007       0.0035       0.0049        0.0063
ACRES                                          0.0000       0.0000       0.0000       0.0000        0.0000
Precision for the Gaussian observations    18409.4864   18250.5308    1360.5198   13057.4525    66442.6714
Precision for TP                              92.9770      77.1663      18.0610      71.3532      297.7739
Precision for SDD                             67.6756      31.9810      26.1615      60.8625      148.6798
Precision for GDD                             15.0452       5.4372       7.0740      14.1594       28.1797
Precision for RICH                           575.6492     379.8899     143.4811     481.5618     1564.3583
Precision for CNTY                            17.3350       1.0539      15.3151      17.3206       19.4529
Phi for CNTY                                   0.7941       0.0234       0.7462       0.7947        0.8382
Precision for AERCODE.id                   19387.5176   18707.2829    1434.1851   13942.5922    68564.0110
Precision for AERCODE.id2                  32023.1182   27478.1455    4554.4349   24499.2638   104475.2154

[[2]]


Table: Model Diagnostic Metrics

       DIC        CPO     MSE          R2
----------  ---------  ------  ----------
 -8804.182   3053.256   4e-07   0.9975864
[[1]]


[[2]]

[[1]]

[[1]]

[[1]]

[[1]]

Sensitivity to Controls

The next set of three models tests the stability of our results given exclusion or inclusion of the linear controls. If the results are stable we can be assured that any ommitted variables are not correlated with those variables.

MODEL without PERC_IRR

This model removes the percent irrigated area control. The functional response for diversity shifts to show more pronounced effects (greater increasing slope) on yield when the irrigation control is removed. This may indicate that there may be ommitted variables that are related to irrigation, and suggests that incomplete consideration of water access may lead to incorrect attribution of effects to diversity. Note, however that the shift in the response is not statistically significant (the 95% credibility bands overlap with the primary model). In addition, the precipitation response curve shifts slightly for low values of TP (this shift is not statistically significant).

[[1]]


Table: Summary Table of Model Estimates

                                                 mean          sd   0.025quant     0.5quant   0.975quant
----------------------------------------  -----------  ----------  -----------  -----------  -----------
(Intercept)                                    4.2603      0.0240       4.2130       4.2603       4.3073
ACRES                                          0.0000      0.0000       0.0000       0.0000       0.0000
Precision for the Gaussian observations       29.8869      0.4702      28.9167      29.9079      30.7588
Precision for TP                             321.7871    156.5944     111.5520     291.4390     710.3334
Precision for SDD                             19.5531      5.5115      11.1095      18.7420      32.6049
Precision for GDD                            119.4451     48.4787      52.0000     110.4213     239.0363
Precision for RICH                           307.8324    131.9028     128.6688     282.0456     635.5308
Precision for CNTY                            20.3590      1.0123      18.3681      20.3652      22.3486
Phi for CNTY                                   0.9991      0.0016       0.9945       0.9998       1.0000
Precision for AERCODE.id                     347.8639     83.8819     215.5822     336.6097     543.5932
Precision for AERCODE.id2                  10169.7726   2162.1384    6395.5188   10030.4679   14846.4144

[[2]]


Table: Model Diagnostic Metrics

      DIC        CPO        MSE         R2
---------  ---------  ---------  ---------
 -4677.75   2158.998   19425.14   11.37197
[[1]]


[[2]]

[[1]]

[[1]]

[[1]]

[[1]]

MODEL without ACRES

This model removes the cultivated acres control.The diversity functional response is not effected by the inclusion/exclusion of the control for agricultural dominance.

[[1]]


Table: Summary Table of Model Estimates

                                                 mean          sd   0.025quant     0.5quant   0.975quant
----------------------------------------  -----------  ----------  -----------  -----------  -----------
(Intercept)                                    4.1897      0.0221       4.1460       4.1898       4.2330
PERC_IRR                                       0.0091      0.0006       0.0080       0.0091       0.0103
Precision for the Gaussian observations       29.8919      0.4514      29.0052      29.8918      30.7832
Precision for TP                             354.8293    179.4273     124.0014     316.6161     809.9875
Precision for SDD                             20.7209      5.5522      11.7782      20.0731      33.4619
Precision for GDD                            122.4016     47.1594      48.9967     116.6600     231.1097
Precision for RICH                          1157.9762    610.4254     400.5112    1019.4567    2723.3939
Precision for CNTY                            21.8925      1.3344      19.0715      21.9978      24.2324
Phi for CNTY                                   0.9995      0.0010       0.9964       1.0000       1.0000
Precision for AERCODE.id                     421.9838    100.7885     246.6001     415.5171     638.2990
Precision for AERCODE.id2                  11097.3847   2421.3413    6842.9250   10950.8272   16296.1950

[[2]]


Table: Model Diagnostic Metrics

      DIC        CPO        MSE         R2
---------  ---------  ---------  ---------
 -4742.39   2204.842   19425.11   11.37197
[[1]]


[[2]]

[[1]]

[[1]]

[[1]]

[[1]]

MODEL without ACRES and PERC_IRR

This model removes the cultivated acres control and the percent irrigated area control. Results are similar to those seen for the model without PERC_IRR.

[[1]]


[[2]]

[[1]]

[[1]]

[[1]]

[[1]]

FUNCTIONAL RESPONSE CONSISTENCY CHECKS

The next set of results shows the output from very simple models with only the predictor variables of interest to see if we identify the same types of relationships as seen in the full model. The goal here is to determine if the shape of the functional responses in the full model are generally stable and hence likely not the result of confounding or misidentification.

MODEL with only SDI

[[1]]

MODEL with only TP

[[1]]

MODEL with only SDD

[[1]]

MODEL with only GDD

[[1]]

Bayesian Prior Sensitivity Checks

This final set of models examines the sensitivity of the model results to choice of Bayesian prior.

Model with Default Priors

Employs the default, generally uninformative priors assigned by R-INLA. Scale-model is TRUE for all random walk and BYM effects. This model provides results very similar to the primary model reported. Minor differences in the functional response curves for RICH and GDD. Precision estimates for random walk variables slightly higher than in the primary model.

[[1]]


[[2]]

[[1]]

[[1]]

[[1]]

[[1]]

[[1]]


Table: Summary Table of Model Estimates

                                                 mean          sd   0.025quant     0.5quant   0.975quant
----------------------------------------  -----------  ----------  -----------  -----------  -----------
(Intercept)                                    4.1765      0.0239       4.1299       4.1764       4.2235
PERC_IRR                                       0.0084      0.0006       0.0073       0.0084       0.0096
ACRES                                          0.0000      0.0000       0.0000       0.0000       0.0000
Precision for the Gaussian observations       30.0115      0.4560      29.1008      30.0189      30.8905
Precision for TP                             358.5003    171.1638     122.8435     327.2547     780.9337
Precision for SDD                             21.0115      5.9696      12.0962      20.0540      35.2562
Precision for GDD                            119.0484     45.5667      50.0529     112.7273     226.1163
Precision for RICH                          1080.0021    606.7912     369.6890     931.0314    2657.2037
Precision for CNTY (iid component)          5152.5083   2726.5090    1752.4499    4539.3968   12146.9648
Precision for CNTY (spatial component)        22.5680      1.1755      20.4189      22.5060      25.0383
Precision for AERCODE.id                     405.5901     94.1879     249.5245     395.9420     618.2865
Precision for AERCODE.id2                  10289.3832   2295.4012    6609.5291   10000.3993   15595.0974

[[2]]


Table: Model Diagnostic Metrics

       DIC        CPO       MSE         R2
----------  ---------  --------  ---------
 -4764.301   2211.742   19425.1   11.37197

Reduced Precision Fixed Effect Prior

Employs the default, generally uninformative priors assigned by R-INLA for random effects. Scale-model is TRUE for all random walk and BYM effects. The fixed effect is given a reduced precision (more uniformative) prior.Results are not significntly differenct from the preceding model and are again consistent with the primary model reported.

[[1]]


[[2]]

[[1]]

[[1]]

[[1]]

[[1]]

[[1]]


Table: Summary Table of Model Estimates

                                                 mean          sd   0.025quant    0.5quant   0.975quant
----------------------------------------  -----------  ----------  -----------  ----------  -----------
(Intercept)                                    4.1745      0.0226       4.1301      4.1745       4.2188
PERC_IRR                                       0.0085      0.0006       0.0073      0.0085       0.0097
ACRES                                          0.0000      0.0000       0.0000      0.0000       0.0000
Precision for the Gaussian observations       29.9543      0.4526      29.0616     29.9557      30.8434
Precision for TP                             440.3373    202.2692     159.7194    403.7628     937.3097
Precision for SDD                             21.4795      5.8026      12.0758     20.8382      34.6499
Precision for GDD                            135.0230     51.1584      56.2287    128.3811     253.5859
Precision for RICH                          1498.6689    862.0813     478.5554   1290.7853    3738.3003
Precision for CNTY                            22.4263      1.1629      20.1179     22.4472      24.6758
Phi for CNTY                                   0.9993      0.0014       0.9953      0.9999       1.0000
Precision for AERCODE.id                     392.6931     92.6895     244.4201    381.0781     606.6409
Precision for AERCODE.id2                  10283.1709   2293.0152    6624.9205   9987.7102   15594.7849

[[2]]


Table: Model Diagnostic Metrics

       DIC        CPO       MSE         R2
----------  ---------  --------  ---------
 -4763.382   2214.372   19425.1   11.37197

Penalized Complexity Prior for Spatial Effects

Employs the default settings assigned by R-INLA for a penalized complexity BYM prior. Scale-model is TRUE for all random walk and BYM effects. The fixed effect is given the reduced precision (more uniformative) prior. Results similar to preceding models and consistent with the primary model. Precision for spatial effects slightly higher than in preceding models.

[[1]]


[[2]]

[[1]]

[[1]]

[[1]]

[[1]]

[[1]]


Table: Summary Table of Model Estimates

                                                 mean          sd   0.025quant    0.5quant   0.975quant
----------------------------------------  -----------  ----------  -----------  ----------  -----------
(Intercept)                                    4.1735      0.0228       4.1291      4.1734       4.2185
PERC_IRR                                       0.0085      0.0006       0.0073      0.0085       0.0097
ACRES                                          0.0000      0.0000       0.0000      0.0000       0.0000
Precision for the Gaussian observations       29.9599      0.4523      29.0848     29.9552      30.8618
Precision for TP                             430.6713    213.0798     158.7570    384.1701     972.8315
Precision for SDD                             21.6887      5.8284      12.2329     21.0480      34.9064
Precision for GDD                            131.4276     53.7475      58.0876    120.8910     265.4927
Precision for RICH                          1468.0367    838.6277     469.3860   1267.5172    3643.4411
Precision for CNTY                            22.3931      1.1333      20.2002     22.3848      24.6474
Phi for CNTY                                   0.9993      0.0013       0.9957      0.9999       1.0000
Precision for AERCODE.id                     405.2712     94.1280     251.0549    394.9229     619.5115
Precision for AERCODE.id2                  10135.7722   2387.8580    6496.5900   9767.0419   15793.3751

[[2]]


Table: Model Diagnostic Metrics

       DIC        CPO       MSE         R2
----------  ---------  --------  ---------
 -4764.343   2214.219   19425.1   11.37197

Penalized Complexity Prior for Random Walk Effects

Employs the default settings assigned by R-INLA for a random walk (order 1) prior (employs standard deviation of the dependent variable as a scaling factor). Scale-model is TRUE for all random walk and BYM effects. The fixed effect is given the reduced precision (more uniformative) prior.Results are not signficantly different from primary model reported.

[[1]]


[[2]]

[[1]]

[[1]]

[[1]]

[[1]]

[[1]]


Table: Summary Table of Model Estimates

                                                 mean          sd   0.025quant     0.5quant   0.975quant
----------------------------------------  -----------  ----------  -----------  -----------  -----------
(Intercept)                                    4.1765      0.0239       4.1299       4.1764       4.2235
PERC_IRR                                       0.0084      0.0006       0.0073       0.0084       0.0096
ACRES                                          0.0000      0.0000       0.0000       0.0000       0.0000
Precision for the Gaussian observations       30.0115      0.4560      29.1008      30.0189      30.8905
Precision for TP                             358.5003    171.1638     122.8435     327.2547     780.9337
Precision for SDD                             21.0115      5.9696      12.0962      20.0540      35.2562
Precision for GDD                            119.0484     45.5667      50.0529     112.7273     226.1163
Precision for RICH                          1080.0021    606.7912     369.6890     931.0314    2657.2037
Precision for CNTY (iid component)          5152.5083   2726.5090    1752.4499    4539.3968   12146.9648
Precision for CNTY (spatial component)        22.5680      1.1755      20.4189      22.5060      25.0383
Precision for AERCODE.id                     405.5901     94.1879     249.5245     395.9420     618.2865
Precision for AERCODE.id2                  10289.3832   2295.4012    6609.5291   10000.3993   15595.0974

[[2]]


Table: Model Diagnostic Metrics

       DIC        CPO       MSE         R2
----------  ---------  --------  ---------
 -4764.301   2211.742   19425.1   11.37197